1. Field of the Invention
This application relates to the acquisition and processing of images that display the full range of gray shades that appear in a physical scene, often referred to as a “High Dynamic Range” or “HDR” image. More particularly it relates to a system and method for the image capture and processing of a HDR image in a digital image capture device such as a consumer grade digital camera.
2. Discussion of Related Art
Images captured by digital cameras are most commonly Low Dynamic Range (LDR) images, in which each image pixel comprises a limited number of digital bits per color. The number of digital bits per pixel is called the digital pixel bit width value. This number is commonly 8 bits. Such 8-bit pixels can be used to form an image with 256 different gray levels for each color at each pixel location. In a LDR image of a scene, shadow areas of the scene are depicted as being completely black (black saturation), bright sunlit areas of the scene are depicted as being completely white (white saturation), and scene areas in between are shown in a range of gray shades. A High Dynamic Range (HDR) image is one that has digital pixel bit width values of greater than 8 bits; 16 bits per pixel is a possible value. In such an image the full range of gray shades that appear in a physical scene can be displayed. These gray shades provide image details that are present in the scene's shadow regions, highlight regions and mid tone regions that are missing from the LDR image. Thus, in an HDR image, scene details are present in image dark areas that are in shadow due to their proximity next to tall buildings and beneath trees, in light areas directly illuminated by bright sunlight, as well as in mid-illumination areas that are lighted between these two extremes.
An HDR image can be captured by acquiring multiple LDR images of a scene that are captured at different exposure levels. These multiple LDR images are called a bracketed exposed image series. A low exposure level will properly capture the gray shades in scene areas fully illuminated by bright sunlight and a high exposure level will properly capture the gray shades in scene areas completely shielded from the sun and sky by buildings and trees. However, at the low exposure level the areas of the scene in shadow will be completely black, in black saturation, and show no detail, and the midtone areas will lose detail. Further, at the high exposure level, the highlights of the scene will be completely white, in white saturation, and show no detail, and the mid-tone areas will again lose detail. Thus, a third, mid exposure level image, which properly captures mid level gray shades, is often acquired as well. By mixing these three LDR images, an HDR image can be generated that depicts the full gray scale range of the scene.
Deriving a HDR image from a bracketed exposed image series currently requires a complex implementation that employs an expensive computational engine. This is due to the need to perform three separate processing operations to properly mix the bracketed exposed image series into a single HDR image, and a fourth to convert the resulting image, which is now composed of pixels with digital pixel bit width values of greater than 8 bits per color, into one that can be displayed on commonly available 8 bit per pixel per color displays. These four processing operations are:
“Image Registration” for accurately aligning the multiple images one to another;
“Image Mixing” for blending the multiple images together with the proper weighting;
“Ghost Removal” for removing location shifted replications of scene objects, or “ghosts”, that would appear in the mixed HDR image, due to the movement of these objects over the time the multiple images were acquired; and
“Tone Mapping” for preparing the final HDR image for presentation on conventional displays that are limited to displaying 8 bit per pixel per color image pixels.
Executing these four processing operations requires the performance of a large number of floating point operations over a short period of time, as can be seen from a review of “High Dynamic Range Imaging Acquisition, Display, and Image-Based Lighting”, by authors Erik Reinhard, Sumanta Pattanaik, Greg Ward and Paul Debevec, published by Morgan Kaufmann Publishers, copyright 2005 by Elsevier, Inc. This is especially the case for the image mixing and ghost removal processing operations. Thus, powerful and expensive computational engines (Central Processing Units or CPUs) need to be used. Their expense can possibly be tolerated for professional digital camera use, but for inexpensive “Point and Shoot” digital cameras, which incorporate limited processing power CPUs, they represent an impractical solution.
An HDR image can be created from a bracketed exposed image series captured by an inexpensive digital camera by uploading the image series from the camera to a general purpose computer, such as Personal Computer (PC). An image processing application, such as Adobe Photoshop, can be used to perform the required complex HDR image combining process on a desktop. This approach is not efficient or convenient and does not meet demands to reconstruct an HDR image on the camera's built-in display shortly after its capture.
Thus there exists a need for an in-camera method and apparatus that can rapidly create a HDR image from a bracketed exposed image series, and display it on the camera's built-in display shortly after capture, using a limited processing power CPU.